Information processing device, portable terminal, control device, and non-transitory computer readable medium

ABSTRACT

An information processing device includes a retaining unit and an image display. The retaining unit retains image-formation-related information of an image forming apparatus that forms an image. The image display causes a display section of a portable terminal having the display section and an image capturing section to display an image for assisting with an operation of the image forming apparatus based on a captured image result of the image capturing section and the image-formation-related information retained in the retaining unit.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.15/592,635 filed May 11, 2017, which is based on and claims priorityunder 35 USC 119 from Japanese Patent Application No. 2016-160219 filedAug. 17, 2016 and Japanese Patent Application No. 2016-164982 filed Aug.25, 2016. The contents of the above applications are incorporated hereinby reference.

BACKGROUND Technical Field

The present invention relates to information processing devices, mobileterminals, control devices, and non-transitory computer readable media.

SUMMARY

According to an aspect of the invention, there is provided aninformation processing device including a retaining unit and an imagedisplay. The retaining unit retains image-formation-related informationof an image forming apparatus that forms an image. The image displaycauses a display section of a portable terminal having the displaysection and an image capturing section to display an image for assistingwith an operation of the image forming apparatus based on a capturedimage result of the image capturing section and theimage-formation-related information retained in the retaining unit.

BRIEF DESCRIPTION OF THE DRAWINGS

An exemplary embodiment of the present invention will be described indetail based on the following figures, wherein:

FIG. 1 illustrates an example of the overall configuration of an imageforming system according to an exemplary embodiment;

FIG. 2 illustrates the internal structure of an image forming apparatusaccording to this exemplary embodiment;

FIG. 3 illustrates the internal structure of an inverting deviceaccording to this exemplary embodiment, as viewed from the direction ofarrow III shown in FIG. 2;

FIG. 4 schematically illustrates the configuration of a wearableterminal according to this exemplary embodiment;

FIG. 5 is a block diagram illustrating a functional configuration of theimage forming system according to this exemplary embodiment;

FIG. 6 is a flowchart illustrating a process performed by a controllerof the wearable terminal according to this exemplary embodiment;

FIG. 7 is a flowchart illustrating a process performed by a print serveraccording to this exemplary embodiment;

FIGS. 8A to 8C illustrate examples of assistance images displayed ondisplay sections of the wearable terminal; and

FIGS. 9A to 9C illustrate examples of images displayed on the displaysections of the wearable terminal when an error occurs in the imageforming apparatus.

DETAILED DESCRIPTION

An exemplary embodiment of the present invention will be described belowwith reference to the appended drawings.

Configuration of Image Forming System

FIG. 1 illustrates an example of the overall configuration of an imageforming system 1 according to an exemplary embodiment.

As shown in FIG. 1, the image forming system 1 according to thisexemplary embodiment includes a print server 3 that manages, forexample, a print job, which is a process based on a print command (i.e.,output command), and an image forming apparatus 5 that serves as anexample of a target object to be operated by an operator and thatexecutes a printing process (i.e., image forming process) on continuouspaper S under the control of the print server 3. The image formingsystem 1 also includes a wearable terminal 7 that is used by being wornby the operator who performs various types of operations on the imageforming apparatus 5 and that serves as an example of a portable terminaldisplaying, for example, an operational assistance image for assistingwith the various types of operations performed by the operator.

In the image forming system 1, the print server 3, the image formingapparatus 5, and the wearable terminal 7 are connected to one another ina communicable manner via, for example, a wireless communication line.

Configuration of Image Forming Apparatus

FIG. 2 illustrates the internal structure of the image forming apparatus5 according to this exemplary embodiment. The image forming apparatus 5according to this exemplary embodiment forms an image onto thetransported continuous paper S. In the image forming apparatus 5according to this exemplary embodiment, the continuous paper S istransported at a high speed of, for example, about 100 meters per minuteso as to allow for high-speed printing of several thousands of pages perminute. The continuous paper S in this exemplary embodiment refers topaper that is continuous in the transport direction in the image formingapparatus 5. The continuous paper S may be, for example, perforated orfolded so long as it is continuous in the transport direction.

As shown in FIGS. 1 and 2, the image forming apparatus 5 is acombination of multiple devices. Specifically, the image formingapparatus 5 includes a first image forming device 51 that forms an imageonto one face of the continuous paper S and a second image formingdevice 52 that forms an image onto one face or the other face of thecontinuous paper S on which the image is formed by the first imageforming device 51. The image forming apparatus 5 also includes a paperfeeding device 53 that feeds the continuous paper S to the first imageforming device 51 and a collecting device 54 that collects thecontinuous paper S by winding it up after the image is formed thereon bythe second image forming device 52. Furthermore, the image formingapparatus 5 includes an inverting device 55 that inverts the front andrear faces of the continuous paper S, where appropriate, having theimage formed thereon by the first image forming device 51 and transportsthe continuous paper S to the second image forming device 52. Moreover,the image forming apparatus 5 includes multiple relay units (not shown)that relay between the paper feeding device 53 and the first imageforming device 51, between the first image forming device 51 and theinverting device 55, between the inverting device 55 and the secondimage forming device 52, and between the second image forming device 52and the collecting device 54.

The image forming apparatus 5 includes a display panel 56 (see FIG. 1)that receives an operation input by the operator and displaysinformation to the operator, and also includes a controller 60 (see FIG.5 to be described later) as an example of a control device that controlsthe operation of each device in the image forming apparatus 5 and thatcommunicates with the print server 3.

As shown in FIGS. 1 and 2, the first image forming device 51 includes afirst image forming unit 511 that forms an image onto one face of thetransported continuous paper S and a first drying unit 512 that driesthe continuous paper S having the image formed thereon by the firstimage forming unit 511. Furthermore, the first image forming device 51includes multiple transport rollers 513 that transport the continuouspaper S. Moreover, the first image forming device 51 includes anoperation panel 515 that receives an operation when, for example, theoperator loads the continuous paper S into the first image formingdevice 51. The first image forming device 51 also includes a housing 516that accommodates, for example, the first image forming unit 511, thefirst drying unit 512, and the transport rollers 513. As shown in FIG.1, the housing 516 has an openable-closable door 516 a that may beopened by, for example, the operator when loading the continuous paperS.

The first image forming unit 511 forms an image onto the continuouspaper S in accordance with a so-called inkjet method. Specifically, thefirst image forming unit 511 has an inkjet head (not shown) havingmultiple nozzles that eject ink therefrom as liquid droplets.

The first drying unit 512 heats the ink ejected on the continuous paperS by the first image forming unit 511 so as to dry the ink.Specifically, the first drying unit 512 includes a heater (not shown)that generates heat by being supplied with electricity.

The second image forming device 52 has a configuration similar to thatof the first image forming device 51. Specifically, the second imageforming device 52 includes a second image forming unit 521 that has aninkjet head and that forms an image onto the continuous paper S inaccordance with the so-called inkjet method, and also includes a seconddrying unit 522 that heats ink ejected on the continuous paper S by thesecond image forming unit 521 so as to dry the ink. The second imageforming device 52 also includes multiple transport rollers 523 thattransport the continuous paper S. Furthermore, the second image formingdevice 52 includes an operation panel 525 that receives an operationwhen, for example, the operator loads the continuous paper S into thesecond image forming device 52. The second image forming device 52 alsoincludes a housing 526 that has an openable-closable door 526 a and thataccommodates, for example, the second image forming unit 521, the seconddrying unit 522, and the transport rollers 523.

The paper feeding device 53 feeds the roll-wound continuous paper S tothe first image forming device 51 via a replay unit while unwinding thecontinuous paper S. The paper feeding device 53 includes a rotatableroller member 531 that supports the roll-wound continuous paper S andalso includes multiple transport rollers 532 that transport thecontinuous paper S unwound from the roller member 531 toward the firstimage forming device 51. Furthermore, the paper feeding device 53includes an operation panel 535 that receives an operation performed bythe operator when feeding the continuous paper S. Moreover, the paperfeeding device 53 also includes a housing 536 that has anopenable-closable door 536 a and that accommodates the roller member531, the transport rollers 532, and the continuous paper S supported bythe roller member 531.

The collecting device 54 collects the continuous paper S having theimage formed thereon by the second image forming device 52 while windingup the continuous paper S into the shape of a roll. Specifically, thecollecting device 54 includes a rotatable roller member 541 that windsup the continuous paper S into the shape of a roll and also includesmultiple transport rollers 542 that transport the continuous paper Sreleased from the second image forming device 52 toward the rollermember 541. Furthermore, the collecting device 54 includes an operationpanel 545 that receives an operation when the operator loads thecontinuous paper S into the collecting device 54. Moreover, thecollecting device 54 also includes a housing 546 that has anopenable-closable door 546 a and that accommodates the roller member541, the transport rollers 542, and the continuous paper S supported bythe roller member 541.

In the image forming apparatus 5 according to this exemplary embodiment,the paper feeding device 53 feeds the roll-wound continuous paper Swhile unwinding it, and the collecting device 54 collects the continuouspaper S while winding it up into the shape of a roll. Alternatively, thetype of continuous paper S is not limited to this. For example,continuous paper S accordion-folded along perforations may be fed whilebeing unfolded by the paper feeding device 53, and may be collectedwhile being accordion-folded along the perforations by the collectingdevice 54.

The inverting device 55 inverts the front and rear faces of thecontinuous paper S, where appropriate, having an image formed thereon bythe first image forming device 51 and transports the continuous paper Sto the second image forming device 52. The inverting device 55 includesmultiple inverting rollers (i.e., a first inverting roller 551 to afifth inverting roller 555, see FIG. 3 to be described below) that guidethe continuous paper S.

FIG. 3 illustrates the internal structure of the inverting device 55according to this exemplary embodiment and corresponds to the diagram ofthe inverting device 55 shown in FIG. 2, as viewed from the direction ofarrow III.

As described above, the inverting device 55 according to this exemplaryembodiment includes five inverting rollers (i.e., the first invertingroller 551 to the fifth inverting roller 555) that guide the continuouspaper S and that invert the front and rear faces of the continuous paperS, where appropriate. Specifically, the inverting device 55 includes thefirst inverting roller 551 that guides the continuous paper S releasedfrom the first image forming device 51 (see FIG. 1) toward thedownstream side, and also includes the second inverting roller 552around which the continuous paper S guided by the first inverting roller551 is wound so as to change the traveling direction of the continuouspaper S by 90°. Furthermore, the inverting device 55 includes the thirdinverting roller 553 around which the continuous paper S guided by thesecond inverting roller 552 is wound so as to change the travelingdirection of the continuous paper S by 180°, and also includes thefourth inverting roller 554 around which the continuous paper S guidedby the third inverting roller 553 is wound so as to change the travelingdirection of the continuous paper S by 90°. Moreover, the invertingdevice 55 includes the fifth inverting roller 555 that transports thecontinuous paper S guided by the fourth inverting roller 554 toward thesecond image forming device 52 (see FIG. 2).

As shown in FIG. 2, in the image forming apparatus 5 according to thisexemplary embodiment, the first image forming device 51, the secondimage forming device 52, the paper feeding device 53, the collectingdevice 54, and the inverting device 55 are each provided with a mark Mthat is visually recognizable at the wearable terminal 7 by the operatorand that is used for specifying a region (unit) of the image formingapparatus 5.

In the image forming apparatus 5, the marks M are provided inside thehousings of the individual devices. For example, in the first imageforming device 51, a mark M attached to each unit is provided inside thehousing 516. When the openable-closable door 516 a of the housing 516 isopened, the marks M are exposed to the outside so as to be visuallyrecognizable.

In the image forming system 1 according to this exemplary embodiment, aunit of the image forming apparatus 5 visually recognized by theoperator is specified by detecting and analyzing an image of a mark Mcaptured by an image capturing section 72 of the wearable terminal 7.Units of the image forming apparatus 5 in this exemplary embodimentrefer to component units of the image forming apparatus 5 that aredifferentiated from one another based on the functions and positionsthereof. For example, the units of the image forming apparatus 5correspond to the first image forming device 51, the second imageforming device 52, the paper feeding device 53, the collecting device54, the inverting device 55, and the relay units.

Although a unit visually recognized through the wearable terminal 7 bythe user is specified based on a unit included in an image captured bythe image capturing section (i.e., based on an analysis result of theimage captured by the image capturing section 72) in this example, theunit may be specified by another method. For example, an ID(identification information) may be given to each unit and betransmitted to the wearable terminal 7 by using Bluetooth (registeredtrademark) Low Energy (BLE), and the unit may be specified by readingthe ID of the unit by using a sensor.

Configuration of Wearable Terminal

FIG. 4 schematically illustrates the configuration of the wearableterminal 7 according to this exemplary embodiment. As shown in FIG. 4,the wearable terminal 7 according to this exemplary embodiment has theshape of eyeglasses in its entirety and is used by being worn on thehead of the operator.

The wearable terminal 7 includes lens sections 71 provided at positionscorresponding to the left and right eyes of the operator when worn, theimage capturing section 72 that captures an image of an entire orpartial region visually recognized through the lens sections 71 by theoperator, and display sections 73 that display an image over the viewvisually recognized through the lens sections 71 by the operator. Thewearable terminal 7 also includes an operation receiving section 74 thatreceives an operation performed by the operator. Furthermore, thewearable terminal 7 includes a controller 80 (see FIG. 5 to be describedlater) that controls each unit.

The image capturing region of the image capturing section 72 may belarger than the region visually recognized through the lens sections 71by the operator so that a larger amount of information is acquired, orthe image capturing region of the image capturing section 72 may besmaller than the region visually recognized through the lens sections 71by the operator so that detailed information is acquired.

The lens sections 71 are composed of a material having opticaltransparency. Thus, the operator wearing the wearable terminal 7 mayvisually recognize the front view through the lens sections 71.

The image capturing section 72 is constituted of a compact camera, suchas a charge coupled device (CCD) camera, and is provided adjacent to oneof the lens sections 71 so as to be oriented in the direction of theline of vision of the operator wearing the wearable terminal 7. Theimage capturing section 72 captures a moving image of the view visuallyrecognized through the lens sections 71 by the operator and outputs thecaptured moving image to the controller 80.

The display sections 73 are formed to overlap the lens sections 71 andare composed of a material having optical transparency when an image isnot displayed thereon. Based on display control performed by thecontroller 80, the display sections 73 display an image over the view(i.e., actual image) visually recognized through the lens sections 71 bythe operator.

The operation receiving section 74 is constituted of a microphone thatreceives sound released by the operator and receives various types ofoperations performed on the wearable terminal 7 based on the soundreleased by the operator.

Functional Configuration of Image Forming System

FIG. 5 is a block diagram illustrating a functional configuration of theimage forming system 1 according to this exemplary embodiment.

As shown in FIG. 5, the print server 3 according to this exemplaryembodiment includes a communication controller 31 that controlscommunication with the controller 80 of the wearable terminal 7 and thecontroller 60 of the image forming apparatus 5. Furthermore, the printserver 3 includes a polling unit 33 as an example of an acquiring unitthat executes a polling operation on the image forming apparatus 5 basedon information acquired from the controller 80 of the wearable terminal7 via the communication controller 31. Moreover, the print server 3 alsoincludes an information storage unit 35 as an example of a retainingunit that stores, for example, information (i.e., device-status settinginformation, which will be described later) acquired as a result of thepolling operation performed by the polling unit 33 and print-job-relatedinformation (i.e., operational information related to an image formingoperation). Furthermore, the print server 3 includes an apparatuscontroller 37 that controls a printing operation performed in the imageforming apparatus 5.

The communication controller 31 acquires an item list that is related tothe device statuses in the image forming apparatus 5 and that is outputfrom the controller 80 of the wearable terminal 7. Furthermore, thecommunication controller 31 outputs information, such as thedevice-status setting information acquired as a result of the pollingoperation performed by the polling unit 33, to the controller 80 of thewearable terminal 7. In the wearable terminal 7, an image is displayedon the display sections 73 based on the information output from thecommunication controller 31. This will be described in detail later. Inother words, the communication controller 31 according to this exemplaryembodiment functions as an image display that causes the displaysections 73 of the wearable terminal 7 to display an image.

The polling unit 33 communicates with the controller 60 of the imageforming apparatus 5 so as to sequentially acquire device-status-relatedsetting information listed in the device-status-related item list.

The information storage unit 35 stores the device-status-related settinginformation acquired by the polling unit 33 and updates the settinginformation, where appropriate.

The apparatus controller 37 controls the printing operation in the imageforming apparatus 5 based on a print job output by, for example, aclient terminal (not shown).

The device statuses in the image forming apparatus 5 will be describedlater.

The controller 80 of the wearable terminal 7 includes a moving-imageanalyzer 81 that analyzes the moving image captured by the imagecapturing section 72. The controller 80 of the wearable terminal 7includes a list creating unit 83 that creates an item list related toinformation (i.e., the device statuses to be described later) necessaryfor creating an image to be displayed on the display sections 73 basedon the analysis result obtained by the moving-image analyzer 81.Moreover, the controller 80 of the wearable terminal 7 includes an imagegenerating unit 85 that generates an assistance image for assisting theoperator in operating each unit of the image forming apparatus 5.Furthermore, the wearable terminal 7 includes a display controller 87that controls the display sections 73 so as to make the display sections73 display the assistance image generated by the image generating unit85. The controller 80 of the wearable terminal 7 also includes acommunication controller 89 that controls communication with the printserver 3.

The moving-image analyzer 81 has a mark detector 811 that detects a markM included in the moving image, which is a captured image result,captured by the image capturing section 72 and a unit specifying unit813 that specifies a unit of the image forming apparatus 5, which isimage-captured by the image capturing section 72, based on the mark Mdetected by the mark detector 811.

The list creating unit 83 has a status storage unit 831 that storesinformation related to items of the statuses of the devices belonging tothe image forming apparatus 5 (i.e., the statuses of units constitutingthe first image forming device 51, the second image forming device 52,the paper feeding device 53, the collecting device 54, and the invertingdevice 55 in this exemplary embodiment), and also has a status selectingunit 833 that selects a device status item related to a unit, which isspecified by the unit specifying unit 813 of the moving-image analyzer81, from device status items stored in the status storage unit 831 basedon the specified unit.

The image generating unit 85 has an image retaining unit 851 thatretains image data related to an assistance image for assisting with anoperation of each unit of the image forming apparatus 5, and also has animage composing unit 853 that composes the image data retained in theimage retaining unit 851 based on the device-status setting informationoutput from the print server 3 so as to generate the assistance imagerelated to the specified unit.

The display controller 87 causes the display sections 73 to display theassistance image generated by the image generating unit 85 over the viewvisually recognized through the lens sections 71 of the wearableterminal 7 by the operator.

The communication controller 89 outputs the device-status-related itemlist created by the list creating unit 83 to the print server 3.Moreover, the communication controller 89 acquires the device-statussetting information output from the print server 3.

In the controller 80 according to this exemplary embodiment, the listcreating unit 83 and the communication controller 89 constitute arequesting unit that requests the print server 3 to output thedevice-status setting information.

Because the image forming apparatus 5 is large in size in the imageforming system 1 according to this exemplary embodiment, the path alongwhich the continuous paper S is guided is long. For example, whenloading the continuous paper S into the image forming apparatus 5, theoperator has to perform this operation while moving around the imageforming apparatus 5 instead of performing the operation at a singlelocation. Moreover, because the image forming apparatus 5 has acomplicated structure and the loading operation of the continuous paperS is also complicated, for example, there is a high possibility that aninexperienced operator may perform the operation erroneously.

Furthermore, the loading operation of the continuous paper S may varydepending on, for example, the settings of the image forming apparatus 5or the print job to be performed in the image forming apparatus 5.However, it is difficult for the operator to constantly ascertain thesettings of the image forming apparatus 5 or the print job to beperformed in the image forming apparatus 5. Therefore, it is difficultfor the operator to determine whether there is an error in the contentsof the operation performed by the operator or an error in the operation.For example, there is a possibility that the image forming apparatus 5may start performing an image forming operation in a state where thereis an error in the loading operation of the continuous paper S. This isnot desirable since the continuous paper S may be cut and the loadingoperation of the continuous paper S may have to be performed again inthe image forming apparatus 5.

In contrast, in the image forming system 1 according to this exemplaryembodiment, the display sections 73 of the wearable terminal 7 worn bythe operator display information related to the operation for loadingthe continuous paper S into the image forming apparatus 5 andinformation related to the device settings set in the image formingapparatus 5 based on the information retained by the print server 3.Thus, the operator may perform the operation while referring to theinformation displayed on the display sections 73 of the wearableterminal 7, thereby reducing an erroneous operation.

A process performed in the image forming system 1 according to thisexemplary embodiment will be described in detail below. The processperformed in the image forming system 1 to be described below isrealized as a result of software and hardware resources operating incooperation with each other. Specifically, central processing units(CPUs) within control computers provided in the wearable terminal 7 andthe print server 3 execute programs for realizing the functions of thewearable terminal 7 and the print server 3, so that these functions arerealized.

Process in Wearable Terminal

First, a process performed by the controller 80 of the wearable terminal7 will be described. FIG. 6 is a flowchart illustrating the processperformed by the controller 80 of the wearable terminal 7 according tothis exemplary embodiment.

When the wearable terminal 7 is activated by the operator, the imagecapturing section 72 starts an image capturing process in step S101. Asmentioned above, in the wearable terminal 7 according to this exemplaryembodiment, an image of the view visually recognized through the lenssections 71 by the operator wearing the wearable terminal 7 is capturedas a moving image by the image capturing section 72.

In step S102, the moving image captured by the image capturing section72 is output to the moving-image analyzer 81.

In step S103, the mark detector 811 in the moving-image analyzer 81detects a mark M from the moving image captured by the image capturingsection 72. If a mark M is not detectable from the moving image (NO instep S103), the process returns to step S101 so as to continuouslyperform the image capturing process by the image capturing section 72,the output of the moving image, and the detection of a mark M.

If a mark M is detected by the mark detector 811 (YES in step S103), theunit specifying unit 813 specifies a unit of the image forming apparatus5 included in the moving image captured by the image capturing section72 in step S104 based on the detected mark M. Specifically, the mark Mdetected by the mark detector 811 is image-analyzed. Then, it isspecified whether any of the units of the image forming apparatus 5 isincluded in the moving image captured by the image capturing section 72based on the type (e.g., shape, color, and pattern) of the mark M. Inthis example, any of the first image forming device 51, the second imageforming device 52, the paper feeding device 53, the collecting device54, and the inverting device 55 is specified as a unit based on the typeof the mark M.

Subsequently, in step S105, an item list related to device statuses inthe image forming apparatus 5 with respect to the unit specified in stepS104 is created by the list creating unit 83. The term “device statusesin the image forming apparatus 5” in this exemplary embodiment refers toinformation about the image forming apparatus 5 set with respect to thecomponents constituting the image forming apparatus 5, the functionsthat the image forming apparatus 5 has, and conditions for a print jobto be executed in the image forming apparatus 5. As mentioned above, theimage forming apparatus 5 according to this exemplary embodiment isconstituted of multiple devices (i.e., the first image forming device51, the second image forming device 52, the paper feeding device 53, thecollecting device 54, and the inverting device 55). Therefore, the imageforming apparatus 5 according to this exemplary embodiment has a largenumber (e.g., about several thousands) of device statuses. Examples ofdevice statuses in the image forming apparatus 5 include thetemperatures and the rotational speeds of the individual components, theenabled/disabled statuses of functions of the components, and the size(width) of the continuous paper S in a print job, but are not limitedthereto in particular.

In step S105, the list creating unit 83 selects a device status relatedto the unit specified in step S104 from among the multiple devicestatuses in the image forming apparatus 5 and creates adevice-status-related item list. In other words, in step S105, the listis created by selecting a device status necessary for creating anassistance image related to the unit specified in step S104 from amongthe multiple device statuses in the image forming apparatus 5.

In step S106, the device-status-related item list created in step S105is output to the print server 3 by the communication controller 89.

Upon acquiring the device-status-related item list from the controller80 (i.e., the communication controller 89) of the wearable terminal 7,the print server 3 executes polling on the image forming apparatus 5based on the device-status-related item list. This will be described indetail later. Thus, the print server 3 acquires setting information setwith respect to the respective device statuses in the item list andoutputs the acquired setting information to the controller 80 of thewearable terminal 7.

Then, in step S107, the communication controller 89 determines whetheror not the device-status-related setting information output from theprint server 3 is received. If the device-status-related settinginformation is not received (NO in step S107), the communicationcontroller 89 waits until the device-status-related setting informationis received.

If the device-status-related setting information is received (YES instep S107), the image generating unit 85 generates an image to bedisplayed on the display sections 73 in step S108 based on the receiveddevice-status-related setting information. In step S108, the imagegenerating unit 85 generates an assistance image for assisting with anoperation of the unit specified in step S102 based on the receiveddevice-status-related setting information. More specifically, based onthe device-status-related setting information, image data related to theunit specified in step S102 is selected from the image data retained inthe image retaining unit 851 of the image generating unit 85. Then, theimage data is composed by the image composing unit 853 so that anassistance image is generated. Although a specific example of theassistance image will be described later, examples of the assistanceimage include a guide image P1 (see FIG. 8A to be described later) usedas guidance for loading the continuous paper S in the specified unit andan image P2 (see FIG. 8A to be described later) related to settinginformation related to an operation of the unit.

In step S109, the display controller 87 causes the display sections 73to display the image generated by the image generating unit 85.

Accordingly, the sequential process in the controller 80 of the wearableterminal 7 is completed.

Process in Print Server

Next, a process performed by the print server 3 will be described. FIG.7 is a flowchart illustrating the process performed by the print server3 according to this exemplary embodiment.

First, in step S201, the print server 3 determines whether or not thedevice-status-related item list of the image forming apparatus 5 isacquired from the controller 80 of the wearable terminal 7 in step S106described above.

If the device-status-related item list is acquired from the controller80 of the wearable terminal 7 (YES in step S201), a polling operation onthe image forming apparatus 5 commences in step S202 based on theacquired device-status-related item list.

Specifically, the polling unit 33 communicates with the controller 60 ofthe image forming apparatus 5 for a predetermined time period (e.g.,several milliseconds). Then, the setting information set in the imageforming apparatus 5 is sequentially acquired with respect to each of thedevice statuses in the device-status-related item list.

In step S203, it is determined whether or not the polling (i.e., theacquisition of the device-status-related setting information) by thepolling unit 33 is performed for the first time.

If the acquisition of the device-status-related setting information bythe polling unit 33 is performed for the first time (YES in step S203),the device-status-related setting information is stored into theinformation storage unit 35 in step S204. Then, in step S205, thecommunication controller 31 outputs the device-status-related settinginformation stored in the information storage unit 35 to the controller80 of the wearable terminal 7.

If the acquisition of the device-status-related setting information bythe polling unit 33 is performed for the second time or onward (NO instep S203), it is determined in step S206 whether or not thedevice-status-related setting information acquired in step S202 haschanged from the device-status-related setting information acquired as aresult of the previous polling operation and stored in the informationstorage unit 35.

Then, if the device-status-related setting information has changed (YESin step S206), the process proceeds to step S207 where thedevice-status-related setting information stored in the informationstorage unit 35 is updated to the device-status-related settinginformation newly acquired in step S202. Then, in step S205, thecommunication controller 31 outputs the updated device-status-relatedsetting information to the controller 80 of the wearable terminal 7.

If the device-status-related setting information has not changed (NO instep S206), the updating and outputting of the device-status-relatedsetting information are not performed, and the sequential process in theprint server 3 ends.

Image Displayed on Display Sections

Next, an example of an assistance image displayed on the displaysections 73 will be described. FIGS. 8A to 8C illustrate examples ofassistance images displayed on the display sections 73 of the wearableterminal 7.

Specifically, FIGS. 8A to 8C illustrate examples of assistance imagesgenerated by the image generating unit 85 in step S108 described aboveand displayed on the display sections 73 by the display controller 87 instep S109.

As shown in FIG. 8A, in this exemplary embodiment, the assistance imagesdisplayed include the guide image P1 used as guidance for an operationof the unit specified in step S102 and the image P2 of the settinginformation related to the operation of the unit included in thedevice-status-related setting information received in step S107.

Examples of the image P2 of the setting information related to theoperation of the unit include information related to the settings ofcomponents constituting the unit, information related toenabled/disabled statuses of the functions that the unit has, andinformation with respect to the components constituting the unit and anerror in the operation performed on the unit.

In this example, information related to B status, D status, A counter, Berror, and E error is displayed as the setting information P2 related tothe operation of the unit.

FIGS. 8B and 8C illustrate examples of guide images P1 displayed on thedisplay sections 73. In this exemplary embodiment, different guideimages P1 are composed in accordance with the device-status-relatedsetting information received in step S107 described above. A composedguide image P1 is displayed in the display sections 73 over the viewvisually recognized through the lens sections 71 by the operator.

For example, in the image forming apparatus 5 according to thisexemplary embodiment, the method of loading the continuous paper S to aroller member (e.g., the transport rollers 513 of the first imageforming device 51, the transport rollers 523 of the second image formingdevice 52, and the inverting rollers 551 to 555 of the inverting device55) varies depending on the width of the continuous paper S.

Therefore, in this exemplary embodiment, a different image is displayedas the guide image P1 used as guidance for loading the continuous paperS to the roller members (i.e., the transport roller 513 of the firstimage forming device 51 in this example) in accordance with informationrelated to the width of the continuous paper S included in thedevice-status-related setting information received in step S107.

For example, in a case where setting information indicating that thewidth of the continuous paper S used in image formation is “half width”is received as the device-status-related setting information, the guideimage P1 shown in FIG. 8B is displayed. Specifically, in the case wherethe width of the continuous paper S is the half width, the guide imageP1 is displayed such that the continuous paper S is wound around thetransport roller 513 from the lower side thereof (i.e., the far side ofFIG. 8B) to the upper side (i.e., the near side of FIG. 8B) thereof, andthe traveling direction of the continuous paper S is subsequentlychanged by 90° so as to be guided leftward, as shown in FIG. 8B.

In a case where setting information indicating that the width of thecontinuous paper S used in image formation is “full width” is receivedas the device-status-related setting information, the guide image P1shown in FIG. 8C is displayed. Specifically, in the case where the widthof the continuous paper S is the full width, the guide image P1 isdisplayed such that the continuous paper S is extended below thetransport roller 513 (i.e., at the far side of FIG. 8C) from right toleft, as shown in FIG. 8C.

Accordingly, in this exemplary embodiment, the image displayed on thedisplay sections 73 is changed in accordance with thedevice-status-related setting information received from the print server3. Thus, the operator performing the operation for loading thecontinuous paper S into the image forming apparatus 5 may perform theoperation while referring to the image displayed on the display sections73 of the wearable terminal 7 even when, for example, not referring to amanual. As a result, even with a complicated operation, such as theloading operation of the continuous paper S, the operation may beperformed readily, as compared with a case where this configuration isnot employed. In addition, by performing the operation while referringto the image displayed in accordance with the device-status-relatedsetting information received from the print server 3, an erroneousoperation in the image forming apparatus 5 may be reduced.

In the examples shown in FIGS. 8A to 8C, the image displayed on thedisplay sections 73 is changed in accordance with the width of thecontinuous paper S as the device-status-related setting information.Alternatively, for example, the image displayed on the display sections73 may be changed in accordance with a combination of multiple pieces ofdevice-status-related setting information.

This exemplary embodiment may achieve reduced traffic and increasedspeed when generating an assistance image (AR image) to be displayed onthe display sections 73.

When displaying the assistance image, image data (i.e., an originalimage serving as a source of the assistance image) is generated bycombining image data (including a three-dimensional model, such aspolygonal data, and icon data) and font data. Moreover, in addition tothe image data and the font data, information (image-formation-relatedinformation), such as the setting information of the image formingapparatus 5 and the setting information of a print job to be output fromthe image forming apparatus 5, is used.

Meanwhile, when these images and information are managed by beingdistributed to multiple devices, the communication traffic increases.Moreover, the intervention of communication leads to lowerimage-generating speed.

In contrast, in this exemplary embodiment of the present invention,information, such as the image data and the font data for generating theassistance image as well as the setting information of the print job, istemporarily stored in the print server 3, and these pieces ofinformation are centrally managed. Thus, reduction in the communicationtraffic and reduction in the image-generating speed caused bycommunication are suppressed. In other words, traffic may be reduced orthe processing time may be shortened, as compared with a case where theassistance-image-related information is managed by multiple devicesconnected to one another via a network. The term “network” refers to acommon network, such as the Internet or an intranet, but does notinclude communication via a dedicated connection line.

As shown in FIG. 8A described above, the error-related information isdisplayed on the display sections 73 of the wearable terminal 7 as thesetting-information image P2 related to an operation of the unit. In theimage forming system 1 according to this exemplary embodiment, when anerror occurs in the image forming apparatus 5 and the error-relatedinformation is displayed as the setting-information image P2 on thedisplay sections 73 of the wearable terminal 7, the error may be removedin accordance with an operation performed on the operation receivingsection 74 (see FIG. 4) by the operator.

FIGS. 9A to 9C illustrate examples of images displayed on the displaysections 73 of the wearable terminal 7 when an error occurs in the imageforming apparatus 5.

Specifically, FIGS. 9A to 9C illustrate examples of assistance imagesgenerated by the image generating unit 85 in step S108 described aboveand displayed on the display sections 73 by the display controller 87 instep S109.

In the example shown in FIG. 9A, the aforementioned guide image P1 andthe setting-information image P2 related to an operation of the unit aredisplayed as assistance images for assisting with the operationperformed by the operator. Furthermore, in the example shown in FIG. 9A,an operation menu image P3 operable by the operator is also displayed.

In this example, B error and E error have occurred in the image formingapparatus 5. As shown in FIG. 9A, B error information and E errorinformation are displayed as the setting-information image P2 based onthe device-status-related setting information received in step S107described above. Moreover, as shown in FIG. 9A, a reset section P31 forresetting the errors is displayed as the operation menu image P3.

In this exemplary embodiment, when B error and E error that haveoccurred in the image forming apparatus 5 are physically removed basedon an operation performed on the image forming apparatus 5 by theoperator, the reset section P31 in the operation menu image P3 becomesselectable, as shown in FIG. 9B.

When the reset section P31 is selected based on an operation performedon the operation receiving section 74 by the operator, B error and Eerror are removed. Specifically, the communication controller 89 in thecontroller 80 of the wearable terminal 7 provides anerror-removal-related notification to the print server 3. Thus,B-error-related information and E-error-related information in thedevice-status-related setting information stored in the informationstorage unit 35 of the print server 3 are updated. As a result, B errorand E error in the image forming apparatus 5 are removed.

When B error and E error in the image forming apparatus 5 are removedand the device-status-related setting information is updated, the imagedisplayed on the display sections 73 of the wearable terminal 7 ischanged accordingly.

Specifically, as shown in FIG. 9C, the B error item and the E error itempreviously displayed as the setting-information image P2 are deleted.Moreover, the reset section P31 item is deleted from the operation menuimage P3.

Accordingly, in the image forming system 1 according to this exemplaryembodiment, when an error occurs in the operation of the image formingapparatus 5, the error may be removed based on an operation performed onthe wearable terminal 7. Thus, for example, it is not necessary toperform an error removing operation using the display panel 56 of theimage forming apparatus 5 after the error is physically removed in theimage forming apparatus 5. Consequently, this may allow for reducedwasteful movement of the operator and may achieve improved ease ofoperation of the image forming apparatus 5.

In the wearable terminal 7 according to this exemplary embodiment, theguide image P1 is displayed on the display sections 73 to overlap theview visually recognized through the lens sections 71 by the operatorwearing the wearable terminal 7 based on control by the displaycontroller 87. Alternatively, for example, the image (actual image) ofthe view captured by the image capturing section 72 may be displayed onthe display sections 73, and the aforementioned guide image P1 may bedisplayed over the actual image displayed on the display sections 73.

Furthermore, in the wearable terminal 7 according to this exemplaryembodiment, a mark M image-captured by the image capturing section 72 isdetected, and a unit in the image forming apparatus 5 is specified basedon the detected mark M. Alternatively, a mark M does not necessarilyhave to be used for specifying a unit in the image forming apparatus 5.Specifically, a unit of the image forming apparatus 5 may be specifiedby pattern-analyzing the position and size of each component and thecontinuous paper S using the moving image of the unit of the imageforming apparatus 5 captured by the image capturing section 72.

Furthermore, although the image to be displayed on the display sections73 of the wearable terminal 7 is composed by the controller 80 of thewearable terminal 7 in this exemplary embodiment, for example, the imagemay be composed by the print server 3, and the composed image may beoutput to the wearable terminal 7.

With regard to other processes, some or all of the processes performedby the controller 80 of the wearable terminal 7 may be performed in theprint server 3. In other words, in this exemplary embodiment, thecontroller 80 of the wearable terminal 7 may constitute an informationprocessing device, the print server 3 may constitute an informationprocessing device, or both the controller 80 of the wearable terminal 7and the print server 3 may constitute an information processing device.

Furthermore, in this exemplary embodiment, the wearable terminal 7 wornand used by the operator is described as an example of a portableterminal displaying an image, such as an assistance image.Alternatively, the portable terminal may be, for example, a smartphone,a portable telephone, a tablet terminal, or a digital camera in place ofthe wearable terminal 7 so long as the portable terminal is, forexample, a medium that is equipped with the display sections 73displaying an image and the image capturing section 72 capturing amoving image and that is worn by the operator for operating the imageforming apparatus 5.

The foregoing description of the exemplary embodiment of the presentinvention has been provided for the purposes of illustration anddescription. It is not intended to be exhaustive or to limit theinvention to the precise forms disclosed. Obviously, many modificationsand variations will be apparent to practitioners skilled in the art. Theembodiment was chosen and described in order to best explain theprinciples of the invention and its practical applications, therebyenabling others skilled in the art to understand the invention forvarious embodiments and with the various modifications as are suited tothe particular use contemplated. It is intended that the scope of theinvention be defined by the following claims and their equivalents.

What is claimed is:
 1. An information processing device for providingassistance to a user to perform an operation on an object, theinformation processing device comprising: a storage that storesinformation related to the object; and a processor executing a programto cause a display of a portable terminal having the display and acamera to display an image for assisting with the operation of theobject based on an identification detected in a captured image result ofthe camera and the stored information, the identification specifying theobject.
 2. The information processing device according to claim 1,wherein the storage stores setting information and operationalinformation as the information, the setting information being set forthe object, the operational information being related to an imageforming operation performed in the object, and wherein the processorfurther executes the program to cause the display of the portableterminal to display the image based on the captured image result, thesetting information, and the operational information.
 3. The informationprocessing device according to claim 1, wherein the processor furtherexecutes the program to acquire the information from the object if animage of the object is captured by the camera of the portable terminal,and wherein the storage stores the information acquired.
 4. Theinformation processing device according to claim 1, wherein theprocessor further executes the program to cause the display to displaythe image for assisting with the operation of the object and an imagerelated to a device setting set in the object.
 5. The informationprocessing device according to claim 1, wherein the image for assistingwith the operation of the object provides operational guidance to a userfor performing the operation in the specified object.
 6. A portableterminal for providing assistance to a user to perform an operation onan object, the portable terminal comprising: a processor executing aprogram to communicate with a control device, which has a storage thatstores information related to the object; a display; and a camera,wherein the display displays an image for assisting with an operation ofthe object based on an identification detected in a captured imageresult of the camera and the information stored in the storage, theidentification specifying the object.
 7. A control device for providingassistance to a user to perform an operation on an object, the controldevice comprising: a processor executing a program to communicate with aportable terminal having a camera and a display; and a storage thatstores information of the object; wherein the processor further executesthe program to receive a captured image result of the camera andtransmits information for assisting with an operation of the object, theinformation being generated based on an identification detected in thecaptured image result and the information stored in the storage, theidentification specifying the object.
 8. A non-transitory computerreadable medium storing a program causing a computer to execute aprocess for providing assistance to a user to perform an operation on anobject, the process comprising: storing information of the object; andcausing a display of a portable terminal having the display and a camerato display an image for assisting with an operation of the object basedon an identification detected in a captured image result of the cameraand the stored information, the identification specifying the object.